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Abstract 

This paper responds to the vision of a new assessment culture, which will be a meaningful integration 
of teaching, learning and assessment. It reviews the advice in the literature about assessment, and 
from there identifies principles for the assessment of the development of higher level abilities in various 
domains. It envisages the student as an active participant in the development of criteria and standards, and 
their consequent use in the making of judgements. The suggested principles are tested out on a recent 
conventional experience that brought the writers together as tutor and student, and on their subsequently 
proposed generic model. 
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Explanation 

This paper has been assembled from three contributions. It began from a teaching person's account by 
Cowan, in which he left gaps for a student's contribution of their experience. However, he first consulted 
students, undergraduate and postgraduate, about the principles he had been formulating. He revised 
these as a result of that Delphic consultation. Having been impressed by Cherry's reflective diary, which 
was more extensive than the norm for her group though not untypical in style, he approached her to be 
the one to fill the gaps in his draft, and to frame her remarks in relation to the principles. Together they 
formulated the introduction, the comments on Cowan's model scheme, and the conclusions they then 
wished to present. Both edited the whole text. 

They use the first person plural ('we') to write about their joint findings and conclusions. Where Diane 
Cherry writes of her own experience, she uses the first person singular ('I') and refers to her tutor as 'John', 
which was her habit in her diary. Where Cowan is reporting his own earlier work, he refers to himself as 
Cowan, to lessen confusion. 

Introduction 

As education concentrates on employability, higher level abilities are increasingly prioritised. Yorke and 
Knight (2006) have stressed the need and means to embed them within the curriculum. Knight (2005:2) 
had already made the case that these complex learning achievements are not -epistemologically speaking 
- measurable, and so require assessment approaches that are radically different from those in routine use, 
and are probably centred on students' claims. This paper presents the further argument that these abilities 
cannot be adequately assessed by teachers or other external parties acting alone. Others cannot be aware 
of certain important aspects of learning and development to which only the learner has access. We thus 
respond to Birenbaum's prediction (1996) of a new assessment culture, meaningfully integrating teaching, 
learning and assessment. 

We subscribe to the views of Scottish universities regarding the attributes that graduates should display, 
seeing these as being developed by self-regulation through self- and peer assessment (Nicol, 2010). We 
advocate harnessing the potential of self-assessment for learning (Boud & Falchikov, 2007; Cowan 1975, 
1978, 1988; Boyd & Cowan, 1986) through the direct involvement of learners in the assessment of their 
higher level capabilities (Dochy & McDowell, 1997; Cowan, 2004a; 2010a). We offer two examples of how 
this can be achieved. 
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Figure 1. contains recent examples of what we mean by higher level learning outcomes. 


Example 1 
A reflective outcome 
(diaries) 


7 take time to identify and seek answers to 
questions about how I undertake tasks, and 
should undertake forthcoming tasks, where an 
answer which I do not already have is likely to 
prove of use to me.' 


Example 3 

An interpersonal outcome 
(group work) 


7 contribute actively to the effective working of 
groups of which I am a member, especially at 
times when these groups encounter difficulties 
of various types.' 


Example 2 
A cognitive outcome 
(designing) 


7 creatively conceive several valid responses to 
problems which are new to me - and objectively 
decide which one to follow .' 


Example 4 
A generic outcome 
(which can be applied in several domains 
and disciplines) 


7 notice without prompting what is significantly 
absent in others' data, writings, statements, or 
proofs of theories - and I decide how important 
these omissions are, and what implications they 
may have' (Wood, 1994). 


Example 5 
An effective outcome 
(tutoring international groups) 


7 contribute actively to the effective working of 
groups of which I am a member, especially at times 
when these groups encounter difficulties of various 
types.' 


Figure 1. Examples of higher level abilities 


Rationale 

We together subscribe to six principles in regard to the assessment of higher level abilities which Cowan 
has substantiated from the literature and with the advice of recent students. 

Six principles for assessing higher level abilities 

PI: The importance of higher level abilities and their assessment should be explicitly emphasised. 

P2: The assessment of desired higher level outcomes will directly influence their development. 

P3: Assessment criteria and standards should be known and understood by students 

P4: Effective learning depends upon adequate provision of formative assessment and feedforward. 

P5: Self-knowledge should inform the assessment of higher level abilities. 

P6: Data provided by learners should be subject to detached corroboration. 

The principles have their origins in his lifelong experience as a facilitator of learner-centred learning 
(Cowan, 2006a), wherein Rogers's advocacy of congruence, empathy and unconditional positive regard 
(Rogers, 1969, 1980) have strongly influenced him since 1971. In such tutor/student relationships, the 
generation of trust (Rogers, 1980; Brookfield, 1990) is powerful in minimising or eliminating power factors, 
especially in regard to assessment and in virtual relationships (Cowan, 2010b). 

PI: The importance of higher level abilities and their assessment should be explicitly emphasised. 

The advent of information and communications technology (ICT) has occasioned a striking transformation 
in priorities, both in higher education and in professional life (Cowan, 2009). Much undergraduate 
provision and subsequent employment formerly centred on the lower-level cognitive demands of knowing, 
understanding, and applying that understanding (Bloom et al., 1956; Anderson et al., 2001). Nowadays 
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ICT has taken over most of these functions (Cowan, 2006b). Thus the education that prepares students for 
professional life (Pennington & Smith, 2002) should concentrate on the higher level cognitive abilities of 
analysis, creativity (Jackson, 2010), problem-solving, and making evaluative judgements (Cowan, 2010a). 
Some time ago the Education for Capability Manifesto (Anon, 1979) declared an expectation that higher 
level interpersonal abilities such as interacting effectively in groups (Cowan, 2005a; Dochy et al., 2007) 
should also be purposefully developed in education for life. Even the long-neglected and demanding 
affective domain is nowadays a subject of attention (Cowan, 2005b; Chiu & Cowan, 2009; Harland & 
Pickering, 2011). Programme designers who seek to develop employable graduates should therefore rate it 
important to feature, and assess in demonstrably valid ways, higher level abilities in several domains. 

P2: The assessment of desired higher level outcomes will directly influence their development. 

The way in which students are assessed fundamentally affects their learning'(QAA, 2006). JISC (2010) 
stresses that learners who 'acquire skills of self-monitoring and self-regulation (for example by assessing 
their own work against defined criteria) prompt deeper and more effective learning.' This influence of the 
'hidden curriculum of assessment' on students' learning has long been appreciated (e.g. Snyder, 1971; 
Gibbs & Simpson, 2004). When Miller and Parlett (1974) classified students as actively cue-seeking, cue¬ 
conscious or cue-deaf, they described students' awareness of the importance of the hidden curriculum, 
and identified their strategic interactions with it. Hence assessment and criteria should be formulated to 
encourage, and then reliably recognise, the desired outcomes - especially including those at a higher level. 

The influence of assessment features as a factor in proactive curriculum development (Stefani, 2004-05). 
Cases now exist where assessment and learning outcomes have been inextricably interwoven since the 
early stages of course design (Cowan, 2004a). This initial and joint consideration of goals and assessment 
is also a feature of self-directed development featuring SMART objectives (Doran, 1981), as in Francis and 
Cowan (2008) and according to Heron (1988). Where learning is completely self-assessed, the directing 
influence of assessment and students' self-chosen criteria has proved to be particularly apparent (Boyd et 
al., 1984; Boyd & Cowan, 1986; Boud & Falchikov, 2007; Boud et al., 2010; Cowan, 2004b). 

'The way in which students are assessed fundamentally affects their learning'(QAA, 2006). JISC (2010) 
stresses that learners who 'acquire skills of self-monitoring and self-regulation (for example by assessing 
their own work against defined criteria) prompt deeper and more effective learning.' This influence of the 
'hidden curriculum of assessment' on students' learning has long been appreciated (e.g. Snyder, 1971; 
Gibbs & Simpson, 2004). When Miller and Parlett (1974) classified students as actively cue-seeking, cue¬ 
conscious or cue-deaf, they described students' awareness of the importance of the hidden curriculum, 
and identified their strategic interactions with it. Hence assessment and criteria should be formulated to 
encourage, and then reliably recognise, the desired outcomes - especially including those at a higher level. 

P3: Assessment criteria and standards should be known and understood by students 

Learners will most effectively improve their abilities if they are clear about the assessment requirements, 
and have a 'feeling' for valid criteria when they are managing and monitoring their development (Sambell 
et al., 2006). Sadler (1989) presented a strong case for learners to have the '...capacity to monitor the 
quality of their own work during actual production'. That capacity depends on the learner's understanding 
and application of the criteria and standards (Cowan, 1975). 

Unfortunately, there is ample current evidence of discrepancies between students' grasp of criteria and 
standards and what their teachers believe has been communicated (Nicol & Milligan, 2006). Criteria and 
standards, especially if carefully specified in academic language, do not commonly convey to the learners 
what is expected of them (QAA, 2003; Hounsell, 2007), and so do not enable all learners to competently 
self-direct and monitor their progress purposefully towards the intended learning outcomes (Sadler, 1989; 
Black & Wiliam, 1998). 
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The importance of commencing to understand from pertinent examples (Cowan, 2006a) was researched 
by Skemp (1971) and Sadler (1989). They reasoned that initial understanding comes from discerning 
meaning from the similarities between positive examples. Thereafter understanding may further develop 
reflectively. The same principle, including the importance of later using non-examples (Skemp, 1971), has 
been re-iterated by others (Markle, 1978; Brookfield, 1990). Where the examples have been produced 
by the learner with whom they are discussed, the learner will already be familiar with, and will better 
understand, the examples' strengths and weaknesses. Cowan's explicit practice is firmly committed to 
establishing the standards for new types of demand from examples (Francis & Cowan, 2008), preferably 
generated by the learners themselves. 

P4: Effective learning depends upon adequate provision of formative assessment and 
feedforward. 

In recent years, the National Student Survey (HEFCE, 2010) has featured strong criticisms of the inadequate 
feedback (which subsumes feedforward) available to students. Students clearly express their desire for 
formative assessment and feedforward, since this informs their interim monitoring judgements and 
prompts their further development (Gibbs and Dunbat-Goddet, 2007; Gibbs, 2010). 

There seem to be grounds to incorporate formative peer assessment with self-directed learning. This leads 
to deeper understanding by both parties, while providing support and generating useful suggestions for 
enhancement - as feedforward (Black et al., 2003). Arrangements which have been called 'peer feedback 
marking' (Falchikov, 1995a and 1995b; Francis & Cowan, 2007) are increasingly popular and effective. 
Biggs (2003) has even asserted that the key to learners' managing enhancement of their development is 
through formative assessment activities that provide peer feedback, which is then driven forward by critical 
reflection (Brockbank & McGill, 1998). 

P5: Self-knowledge should inform the assessment of higher level abilities. 

We together plead this point through argument, rather than from citations. We suggest that only a learner 
has primary access to their personal thoughts and feelings. Only learners know the origin of their creative 
problem-solving - whether it stemmed from an original idea, or from the adaptation of something familiar 
or recalled (Boyd & Cowan, 1986). Only a learner knows the nature and effectiveness of the wrestling in 
which they engaged when they struggled to cope, and worked out how to do so. Only a learner can be 
fully aware of the options which they identified and explored - and the nature of the reasoning on which 
their final choice was founded. 

Assessors cannot make judgements of such personal mental activity from external observation, or by 
devising assessment tasks and marking scales. They need the input of what is initially only known to the 
learner about their learning engagement, before formulating a fully informed and adequate judgement of 
the learning and development. 

P6: Data provided by learners should be subject to detached corroboration. 

Corroboration is a desirable feature in decision-making in social sciences and education, as in Scots (as 
compared to English) law. Burns (1786) memorably stressed the importance of judgements being based 
upon sound factual data rather than opinions, maintaining that: 

'My skill may weel be doubted; but facts are chiels that winna ding, an downa be disputed' 

(tr: facts, like children, will not change their story). The formulation of objective judgements should, 
whenever possible, be based on relevant facts drawn from reliable and corroborated sources (Dowie 
& Elstein, 1988; Brown & Paschoud, 2005). A learner's claims may well have been endorsed with 
"corroborative detail, intended to give artistic verisimilitude to an otherwise bald and unconvincing 
narrative" (Gilbert, 1885). As noted above, the learner's self-knowledge is the origin of claims for 
development of abilities. Nevertheless, sincere but unsubstantiated self-judgements about one's ability or 
performance may prove unsound because of self-delusion, lack of rigorous self-scrutiny, misunderstanding 
of criteria, or simply inadvertent inaccuracy or omission. They should be carefully checked against external 
data. 
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Checking the principles in a particular example 

Diane explores first our own recent (2010) joint experience, when John was a virtual tutor (Cowan, 2010b), 
and Diane a student; and where the conventional course programme within which we worked had been 
designed by others. Where our principles seem to apply, this has been noted in parentheses in Diane's 
account. 

Diane writes (in text which John has not sought to change, other than in assisting to reduce the word 
count and suggesting occasional references : 

I enrolled for the Edinburgh Napier's Postgraduate Certificate (PGC) of Leadership and Excellence. This 
included a work-based learning project related to initial learning on the course, and to my current 
employment. The project module required the submission to the tutor (at an unspecified frequency) 
of a reflective diary, as an integral part of the learning process [P2j. At the half-day induction meeting, 
the course director emphasised [P3] that the use of reflective diaries was to identify, enhance and fulfil 
cognitive, reflective, affective and higher learning needs [PI] (Chiu & Cowan, 2009). The learners should 
be actively engaged with the learning experience and identifying their own learning needs, while giving 
cognisance to their internal, affective and experiential learning. I did not immediately appreciate the 
possible benefits to me. Yet with hindsight, as I will describe, the diary was and continues to be one of the 
most valuable developmental experiences I have ever undertaken. 

Initially, it was difficult for me to find the discipline needed to produce this diary. I put this down to the 
fact that my preferred learning style had been assessed in a previous course as strongly activist/pragmatist, 
with lower scoring in theorist and reflector styles. But I made entries, sometimes every day, to reflect 
on my reading of materials or on meetings attended. The earlier entries were not very focussed, and 
used to offload anxieties and frustrations relating to the project. Quite quickly, entries became a way of 
constructively exploring issues that cropped up during the reading phase - to be remembered, picked up 
and further developed during the writing stage, having meantime been worked on by my 'back brain'. It 
was an uncensored, 'safe' area in which to explore, analyse and debate issues both internally - and, with 
feedback from John, to judge their value, or not, to the final project [P4], 

This project called for a large element of self-directed study. That could have engendered feelings of 
isolation for me. I was more used to traditional university courses, with attendance at lectures and tutorials 
fostering a sense of community and creating a supportive peer network. In a large part, keeping this diary 
and engaging in two-way digital dialogue with John served as a proxy for being able to discuss progress 
informally, face-to-face with my peers. It developed important intrapersonal, academically focussed, skills. I 
felt a sense of loss when the course was over, and the two-way feedback with John was at an end. 

These facilitative interchanges, with John's 'nudging' of me, as I learnt Bruner (1986) had described 
it, prompted my student-directed monitoring and development of relevant abilities, including my self¬ 
management, analysis and objective evaluation, and my reasoned prioritising. Comment and feedforward 
in a challenging but supportive form (Cowan, 1984), was important to me. I submitted my diaries weekly, 
and John was able to gauge my progress - or not. This frank interaction kept my project on track, simply 
because I was receiving quality ongoing support and challenge [P4], 

The essential requirement for this arrangement to succeed was honesty. Tutor and student must trust each 
other enough to be completely frank, and not simply write what the other expects to hear. Some of the 
learning and development that was occurring was initially only known to me [P5j; but importantly it was 
recognised, valued and acted upon by both of us - after I recorded it. 

My learning style is now more reflective, with less likelihood of my beginning my next work project by 
jumping in without theorising about alternative approaches or reflecting on relevant past experiences. 
Overall, I regarded the reflective diaries as so beneficial that I have continued to keep one for any major 
pieces of work I now undertake. I have also encouraged others in the organisation to adopt the practice. 
This reflective discipline can be argued to be the longest lasting, most obvious benefit to my personal 
development that this course brought about [PI ]. 
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Diane reflects 

This course experience (in a traditional form unlike the general scheme which follows) embodied features 
which are mostly in accord with the principles we established earlier. 


PI: The importance of higher level abilities and their assessment was emphasised. 

The whole point of the course was for me to understand, appreciate the importance of, and apply 
leadership skills, tied to achieving quality outcomes for my organisation. It wasn't enough to know what 
these skills were; I had to model them personally and recognise the areas where my behaviour could be 
modified to make me a better leader. Knowing is different from behaving, and the development of this 
emotional intelligence and the devising of a framework for consistent application of those behaviours 
across my workplace was the goal of my project. It was all about transferable skills - you already have a 
head start in any workplace if you are displaying the leadership behaviours critical for success. 

The short-term benefits and the finite nature of the actual project were one thing, but the major 
'takeaway' learning for me came from the use of the diary rather than the specific work-based learning 
within the report. I still use the diary technique, now without John's probing and challenge, for other 
work-based projects, to capture and explore my thoughts and feelings, and to allow for self-critical and 
realistic reflection. It has made me much more introspective, questioning everything I learn, relating it to 
my perception of reality within my workplace. I am much more likely to reflect on how each piece of work, 
project or meeting went, with task outcomes, interaction with other staff and how I behaved/contributed 
being the major factors I consider routinely. I believe this is the ultimate intrapersonal learning outcome to 
be expected from any higher education qualification. 


P2: The assessment of my higher level outcomes directly influenced their development. 

At the half-day induction, the aims and assessment requirements for successful completion of the course 
were made clear to us, both verbally and in the Practitioner Learning Handbook. These covered delivery 
of the written work-based learning project, accompanied by the use of a reflective diary, centred on the 
development and use of higher level abilities, to inform their assessment. Embarking on this course, I had 
no prior knowledge or experience of the use of reflective diaries in this context; but the reasons for using 
one soon became apparent, through my exchanges with John. 


P3:1 well knew and understood the assessment criteria and standards 

(See P2). During and through interaction with John, whom I met only virtually after the induction meeting, 
he used the examples I was producing to offer explanatory comments on how well I matched up to desired 
standards. So I soon developed my feeling for what would be "good" project work and what is "valuable" 
personal development. 


P4: Effective formative assessment and feedforward promoted my learning. 

I would have welcomed peer assessment and feedforward at regular intervals on the report's subject 
matter. It would have been useful if such peer review had been formally incorporated in the programme, 
with a structure that helped students to generate constructive feedforward and cope with receiving it, so 
that it didn't derail or demotivate. However my feedforward came from John's comments on my diaries. 
Initially, I was writing far more in the diary than I was contributing to the actual writing of the project. 

It was important to me to capture and review what I was thinking, especially in relation to the project's 
reading stage, when my thoughts were spiralling in many directions and reaching some conclusions which 
would never make it into the final report. Although at the start I wasn't fully aware of the purpose of the 
reflective diary, I made a clear decision to be absolutely honest in it - describing my fears, any confusion, 
how I actually felt and why - even if it meant I was admitting to feeling unworthy to be on the course. I 
was also able to capture my thinking and my assorted sources; this acted as a useful aide-memoire when it 
came to writing up the project. 
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It felt good to be able to do this 'uncensored', as some of my sources were not always academic, and some 
of the entries were critical of colleagues who were perhaps not behaving in as supportive a manner as I 
had hoped. On reflection, it acted as a safety-valve, and a private space where I could explore, internally 
debate, accept or reject ideas and creative concepts - many of which never made the light of day in the 
final version of my report. 

John did not instruct me. He consistently challenged my thinking - bringing some outside perspective, 
balance and reality to the situation. I do not feel that I would have gained as much from the keeping of the 
reflective diaries if I had written what I thought he wanted to hear. I was aware throughout that he would 
ultimately be making a pass or fail recommendation on my report - but I ignored that when it came to the 
diaries and what they contained. I knew that John would never let me submit a below standard final piece 
of work so I had nothing to lose and everything to gain by being completely honest during the journey to 
get there as described in the diaries. The feedback and feedforward on individual entries made me examine 
my own thinking in an objective way, requiring me to reflect on how I had formulated my thoughts in the 
first place. That prompted and enabled me to scrutinise their validity objectively, which I would certainly 
not have done had I not been keeping the facilitated diary. 

I'm not sure that such facilitation will always be necessary in future projects, as a mature diary writer 
should then display the necessary objectivity and self-critique of their own diary entries. When I look at my 
current diaries, I see myself adopting John's role and picking up on the type of issues that he highlighted. 

I even started to do that while I was writing the later course diaries. I noticed much less probing and 
questioning from John towards the end of the diaries than at the start when I was unused to the process. 
Whatever the source of questions, any claims made in a diary or indeed project should be subject to 
probing - we all have a capacity for self-deception. 


P5: My declared self-knowledge contributed to the assessment of my higher level abilities. 

Only part of the development and exercise of abilities which featured in the diary was overtly apparent in 
my final report which formed the main basis for institutional assessment of my work. Those who assessed 
that report volunteered that they had found much of worth 'between the lines' through having read my 
diaries. This point is surely unquestionable - that only learners themselves can be fully aware of all the 
outcomes of a learning experience. The issue is not simply about what a self-assessing learner produces 
in a report. Assessment of me would have been incompletely informed without the diary and its record of 
reflection and tutorial dialogue. 


P6: As I went along, I was checking the data which Informed my self-judgements. 

John was concentrating on how I was tackling the project, not what I was doing. So in our digital dialogue, 
he was automatically prompting me to check how I was doing things, as we went along. 

With innovatory freedom, we together now jointly propose a valid and reliable general approach, using 
elements from evaluated practice (Cowan, 2006a and 2006b; Francis & Cowan, 2008). This approach 
began in unreported activity by Cowan in the early 1980s, with first-year civil engineers. He was at that 
time seeking cost-effective ways to arrive at valid peer assessment of a year's efforts by 129 students to 
keep and profit from reflective learning journals. He still sets himself the criterion of cost-effective use of 
tutor time. 

A generic model for the assessment of higher level abilities 

1: The tutor provides students with two or three mid-standard examples of comparable student output, 
from an associated discipline and context. Students in groups consider the examples, and identify their 
strengths and weaknesses, formulating clear and objectively worded assessment criteria as they do this. 
The tutor comments when the students do or do not demonstrate understanding of their criteria (as in 
Cowan, 2004c) [P3], 

2: At any stage, an individual student may declare and apply additional criteria, appropriate to course 
aims and their performance, against which they will wish to make claims and self-judgements [P5j. 
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3: On several occasions during the programme, each student summarises claims for the level of their 
performance, or for the development in their ability, as agreed at the outset [P4]; they illustrate their 
claim with perhaps three examples, supported by data [P5], 

4: Wherever possible the student's data should be derived from, or corroborated by, third parties, and not 
merely taken from their personal journals [P6], 

5: Each student completes their self-assessment by explaining, in terms of the criteria, why and to what 
extent they consider their examples commendable, and in what ways [P5], 

6: The student receives facilitative suggestions, preferably anonymously and online, from a constructive 
peer, to strengthen the presentation of their claim - without their peer assessing them. The peer should 
advise if the student has addressed the criteria, if their claims are described in comparable wording, if 
they are adequately supported by data (Ramprasad, 1983), and what could be done to strengthen the 
self-assessment [P4], 

7: The peer's advice, together with the student's responses, should accompany the final claim [P6], 

8: The tutor audits the final self-assessed claim, and the use of the peer's suggestions, noting if there are 
any discrepancies in the way the student has followed the prescribed process [P6], 

9: The student and tutor then discuss, and hopefully agree, the judgement that will be recorded (Taras, 
2001). [P6] 

Joint conclusions 

We have confronted the special challenge of assessing higher level abilities. We have assembled 
principles which emerged for us from the literature; and we have analysed a particular example, within 
a conventional course structure, in these terms. We have then suggested a possible generic and radical 
model for the assessment of higher level abilities. We conclude that the principles we have summarised are 
generally relevant, applicable, and give due weight to the self-knowledge which the learner should bring to 
their assessment. 
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